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FOREWORD 



Extensive research conducted under the Federal-State cooperative test 
research program in the Training and Employment Service has led to the 
development of many tools useful in vocational counseling and placement. 
These tools include aptitude tests, proficiency tests, and noncognitive 
measures based on instruments such a3 interest inventories and biographical 
information blanks. 

The purpose of this series of reports is to provide results of significant 
test research projects \s they are completed. These reports will be of 
interest to users of the tests and to test research personnel in other 
organizations. 

This report was prepared in the Utah Department of Employment Security as 
part of a cooperative research study under the general direction of the 
U.S. Training and Employment Service. It is one of seven prepared by State 
agencies affiliated with the U.S- Training and Employment Service on the 
subject of live and recorded methods of dictation test administration. The 
seven studies conducted from 19 $9 to 1967 are summarized in the USTSS Test 
Research Report No. 28, December 1969. 
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STUDY TO COMPARE RELIABILITY OF PERFORMANCE ON LIVE AND 
RECORDED DICTATION TESTS 



/ 



PURPOSE OF STUDY 

This study was undertaken to compare the reliability of performance on record- 
ed dictation tests and the reliability of performance on these same tests when 
administered "live." 



SAMPLE DESCRIPTION 

The total research sample was drawn from two sources. These were: (1) Uni- 

versity shorthand students and (2) local office job applicants seeking steno- 
graphic positions. Preliminary plans to include employed stenographers and 
business school students enrolled in shorthand courses were eliminated as the 
study progressed. Sample members were administered two equivalent test forms 
either "live" or by tape recording. 

The University sample was composed of 227 students who were nearing completion 
of Business Education Course 112, I ntermediate Shorthand at Brigham Young 
University (BYU) , Provo, Utah. Content of this semester course is described 
in the University Catalog as foll)wsj "Development of shorthand writing and 
control beginning with 60 words per minute to develop a rate of 80 to 100 
words per minute for three minutes on unfamiliar material." Business Education 
111, Elementary Shorthand is a prerequisite. Female instructors teach the 
Business Education 112 Course, but recorded dictation exercises with mostly 
male voices are used extensively in the teaching process. 

Three classes of Business Education Course 112 students were tested. Each of 
the three classes was divided into several sections. Erch section had its 
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own instructor and regularly scheduled classroom and instruction period. 
Classroom facilities for the various sections were comparable. The experi- 
mental test battery was administered on May 11 and May 12, 1966 to students 
in their own sections during their regularly scheduled class periods of 8.00 
a.m., 1:00 p.m. or 2:00 p.m, The method of administration -- "live" or 
"recorded'* -- for each section was selected randomly. Four trained test ex- 
aminers from the Utah Department of Employment Security administered the dic- 
tation tests. Two of these examiners were males and two were females. 

Nine students were dropped from the sample because they did not attempt to 
transcribe nil of the dictation exercises. Two more students were eliminated 
because of Incomplete personnel information. Therefore, the final University 
sample consisted of 216 students all of whom were females except one. This 
sample was about equally divided between the two methods of administration. 
There were 105 students in the "live" group and 111 in the "recorded" group. 

The job applicant sample was obtained from the Professional, Clerical and 
Sales Employment Office (PCS) in Sait Lake City, Utah. Persons seeking 
stenographic positions were administered the experimental test battery in the 
employment office proficiency testing room during regularly scheduled testing 
sessions between June 6 and June 23, 1966. The method of administration was 
changed randomly from day to day, but the method selected for a given day re- 
mained the same for all testing sessions. Two trained female test examiners 
from the Utah Department of Employment Security administered the dictation 
tests to 28 job applicants. Two persons were eliminated from this group. 

One applicant did not complete transcription, and the other was identified as 
a member of the BYU student sample. The final sample of 26 applicants in- 
cluded one male with 14 examinees transcribing "live" dictation tests and 12 
transcribing "recorded** dictation tests. 

6 
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The final research sample of 242 examinees included ten sub-samples- Sam- 
pling characteristics of these sub-samples are summarized in Table 1 below; 



TABLE 1 

Summary of Sampling Characteristics for the Ten Sub-Samples 



Sample 


N 


Source 


Method 


Examiner Sequence* 


Testing Schedule 


01 


tr\ 

CM 


BYU 


Live 


A-4, B-10 


5- 11-66 


1;Q0 p.m. 


02 


35 


BYU 


Live 


C-4,10 


5-11-66 


1:00 p-m. 


03 


29 


BYVJ 


Recorded 


B-4,10 


I 

>— * 
1 

O' 

cr 


2:00 p.m. 


04 


33 


BYU 


Recorded 


> 

1 

X> 

o 


5-11-66 


2;00 p*m. 


05 


18 


BYU 


Recorded 


C-4. 10 


5-11-66 


2:00 p.m. 


06 


34 


BYU 


Live 


B-4 , A-10 


5-12-66 


8:00 a.m. 


07 


11 


BYU 


Live 


C-4,10 


5-12-66 


8:00 a-m. 


08 


31 


BYU 


Recorded 


D-4,10 


5-12-66 


8:00 a.m. 


09 


14 


PCS 


Live 


C-4,10 
E-4,10 
C-4 ,E- 10 
E-4.C-10 


6-6-66 


to 6-23-66 


10 


12 


PCS 


Recorded 


C-4,10 
E-4,10 
C-4, E-10 
E-4 , C-10 


6-6-66 


to 6-23-66 




* A, C, E * Female Examiners 
B and D * Kale Examiners 



4 * USES Dictation Exercise No. 4 
10 * USES Dictation Exercise No. 10 
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Table 2 shows age, education, training, and experience characteristics for the 
combined student sample and the combined job applicant sample. The average 
member of the student sample was relatively young and was nearing the end of 
two years in college. Age range was from 18.0 to 29.2 years with a mean of 
19.9 years. Educational range was from 12 to 17 years with a mean of 13.6 
years. All students had had at least one course in shorthand with an upper 
range of four courses. 



TABLE 2 

Keans, Standard Deviations (SD) , and Ranges for Age, Education, 
Training and Experience for the Combined Samples 





Combined Student Sample 
N=216 






Mean 


SD 


Range 


Age (years) 


19.927 


1.849 


18.0-29.2 


Education (years) 


13.611 


. 926 


12-17 


Training (courses) 


2.273 


.813 


1-4 


Experience (months) 


5.689 


9,824 


0-4 8 


Combined Job Applicant Sample 
N*26 






Mean 


SD 


Range 


Age (years) 


20.558 


5.801 


17.6-42.0 


Education (years) 


12.692 


.86/ 


12-15 


Training (courses) 


2.115 


.640 


1-4 


Experience (months) 


24.692 


66.117 


0-276 




8 



5 



The sample mean for shorthand training was 2.3 course . Some students had 
had no on-job shorthand experience while other , ha worked up to 48 months 
on jobs where shorthand was used. Averse snoiVna <? >r experience was 5.7 
months. Except for shorthand experience, the v. u; nilltv of each of these 
student characteristics v..s limited as evidenced y tt_*ndaid deviation values. 

The combined job applicant sample of 26, although much smaller, was not too 
different from the combined student sample in mean age, » hucatiun, and num- 
ber of training courses . On the average, the job applicant sample was .7 
years older, had .9 years less education, and had completed . 1 fewer shorthand 
courses* However, the average job applicant hud had more than four tiu:*s as 
much on-job shorthand experience The job applicant sample also showed more 
variability in age ard experience than tne student sample. 

For purposes of analysis the eight BYU sub-samples were combined along specifi 
research dimensions. These dimensions .ncluded number or examiners, method 
of odmtnis trat ion, some stenographic experience, and no s tenogiraplvic experieuc 
These configurations are shown below: 



Live Recorded 



One Examiner 


(N = 46) 


One Examiner 


<N = 111) 


Two Examiners 


(N = 59) 


Two Examiners 


(N - 0) 


Some Experience 


(N = 50) 


Some Experience 


Ctf = 42) 


No Experience 


(N « 55) 


No Experience 


(N *■ 69) 


Total 


(N * 105) 


Total 


(H - 111) 
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EXPERIMENTAL TEST BATTERY 

Three USES 80 worths per minute dictation exercises were administered either 
"live" or by tape recording to all members of the research sample. These 
included a Practice Exercise and Exercise No. 4 and Exercise No. 10. All of 
these exercises were taken from the Guide to the Use of Typ ing , Dictation and 
Spelling Tests (hereafter referred to as the Guide) , published by the Bureau 
of Employment Security in December, 1953. 

Exercise sequence was always the same for both "live 11 and "recorded" test 
administration. This sequence was: (i) Administration of Practice Exercise; 

(2) Administration of Exercise No. 4; (3) Transcription of Exercise No. 4; 
t4) Administration of Exercise No. 10; and (5) Transcription of Exercise No. 10* 
The exercises were read verbatim from the Guide by each test examiner when ad- 
ministering "live" dictation tests. "Recorded" dictation tests utilized stan- 
dard tape recordings of these same exercises wnicn were recorded professionally 
by sound engineers using the male voice of an experienced radio broadcaster. 
These standard recordings were played by each examiner when administering 
"recorded" dictation tests. 

All test instruction? were given "live." Standard directions were used when 
administering the dictation exercises, but some information was added to these 
standard instr uc lions to facilitate administration by tape recording. An intro- 
ductory statement explaining the purpose of the study preceded the giving of 
test instructions to all examinees. 




DATA COLLECTION 

Four trained examiners, two male and two female, administered the experimental 
battery to the BYU student sample. Some students were administered the exor- 
cises "live" and some by standard tape recording. Correct timing of "live" 
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exercises was maintained by using a stop watch. Only one examiner was present 
in a student section when the exercises were administered by tape recording. 
Taped exercises were played from a central location and piped into appropriate 
classrooms upon signal from the individual examiner. With tnis arrangement 
the same recorded exercise could be administered simultaneously to more than 
one section whenever the test schedule so required. Either one or two exam- 
iners were present in each student section when the exercises were administered 
,r liveV When two examiners assisted in administration, one examiner gave Exer- 
cise No. 4; and tne other examiner gave Exercise No. 10. The two examiners 
then exchanged exercises when administering dictation tests to a different 
sec t ion . 



Because test administration had to be limited to a 50-minute class pex:od, no 
break was allowed in the testing sequence; ar.d the standard transcription time 
for Exercises4 and 10 was reduced from 20 minutes to 15 minutes each. Transcrip 
tions were typed double space by students while seated at individual typing 
stations. Each exaxninee recorded his name, school, test date, dictation speed, 
method of administration, and name of examiner (s) in the upper right hand 
corner of each transcription sheet. Completed transcriptions were collected 
and maintained in batches by student section. 



Prior to the actual testing period each student was asked to complete a 
mation sheet. This sheet provided space for entering personnel informa 
sucn as name, date of birtn, highest year of education completed, lengt 
sliurtnand training, and length of shorthand work experience. It also a 
the student's personal preferences on method of test administration and 
examiner. All personal and preference data were collected for the fina 
of 21b students. 



n inf-r- 
t ion 
n of 
sked 
sex of 
1 sample 
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Two female examiners administered the tests to the job applicant sample in 
small groups of one to four examinees. One of these examiners was the test- 
ing clerx assigned to the PCS Office, and Lne other examiner was one of those 
who had helped administer dictation exercises to the BYU student sample. The 
two examiners alternated in giving the test batteries. Sometimes one examiner 
gave the entire battery and sometimes the other. On other occasions both 
examiners participated, one examiner giving Exercise No. 4 and the other 
Exercise No. 10. Examiner order was reversed from group to group when both 
examiners participated in test administration. Recorded exercises were played 
monaurally on a Sony Sterecorder "ZOO 11 equipped with tvo speakers spaced about 
4 feet apart. Tone and volume were preset to provide optimum quality of 
sound reproduction. "Live" dictation exercises were administered by having 
the examiner(s) read the test verbatim from the Guide , maintaining correct 
dictation speed by means of a stop watch. 

Transcriptions of shorthand notes were typed double space by applicants while 
seated at individual typing stations. Twenty minutes was allowed for each 
transcription. Each examinee recorded his name, address, test date, dictation 
speed, and method of administration on his transcription sheet. Examiner 
identity was added by the administrator after t he transcriptions were completed. 

Information sneets like those used with thi student Sample were completed after 
testing by each applicant to obtain personnel data and testing preferences. 

Data on age, education, shorthand training, and shorthand work experience were 
collected for all 26 members of the final job applicant sample. Preferences 
for method of test administration and sex of examiner were obtained from all 
but one job applicant, an examinee who was administered the test M iive." 
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DATA PROCESSING 



Data for each of the ten sub-samples were readied for processing by arrang- 
ing Information Sheets and transcription materials alphabetically by examinee 
Surname. After this procedure each dictation exercise was scored twice 
according to "Directions for Scoring Transcrip tion lf appearing in the Guide , 
and the error score was posted on the face of the transcription sheet. 
Transcription materials for each examinee were then matched with his Informa- 
tion Sheet. 

Testing preferences for each examinee were read from Information Sheets and 
posted to work sheets. Percentages of sample response were then computed 
for different sample combinations from tallies of preference categories. 
Personnel data and error scores were posted to ut’ner work tables for key 
punching Into data cards. Postings were checked for accuracy, and all data 
punched into cards were verified. 

Punch cards containing examinee name, sex, examinee number, study number, 
sample number, error scores on Exercises 4 and 10, age, education, shorthand 
training, and shorthand work experience were then used for ADP computation 
of basic statistics for these variables. These statistics included means, 
standard deviations, ranges, Pearson product ‘moment correlations, and relia- 
bility coefficients. Machine processing of data was acr mplishtd for indi- 
vidual and combined samples from appropriate decks of data cards. Hard 
copies were printed for each machine run, and pert! lent tables were prepared 
from these ADP printouts. 

Analycis of variance and tests for significance of difference between means 
and reliability coefficients were computed by means of a desk calculator. 

All of these hand calculations were checked for accuracy. 
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DATA ANA1 YSI3 



Sample Characteristics 

Tables 3,4,5, and 6 snow number of examinees* means, standard deviations, and 
ranges for age, education, shorthand training, and shorthand work experience 
for the ten experimental samples. Analysis of these data indicates that mem- 
bers of the stuaent samples were, on the average, quite similar as far as age 
and education are concerned. They were generally similar in average amount of 
training received but varied considerably in average length of work experience. 
With the exception of work experience the average members ot the two job appli- 
cant samples were very much alike.. lowever, Sample 9 was a somewhat older group 
and exhibited greater variability in age and shorthand wurk experience. 

Aitnough variability in age of one of the job applicant samples (Sample 9) was 
greater than that of the student samples, the mean ages of the two job appli- 
cant samples were about like those of the student samples, (Jn the average, job 
applicant samples had less education than student samples. Mean ?ength o£ 
training for members of job applicant samples was more than mean length of 
training for some student samples but less than mean length of training for 
other student samples. Mean work experience of job applica.it samples appeared 
to be higher than mean wor* - experience of all student samples except Sample 4. 

Although group differences still existed, mean differences among individual 
student samples tended to be 'ess pronounced when members of tne eignt experi- 
mental samples were combined according to type of test administration and 
factor uf work experience, but no real experience difference vas apparent 
between experienced students taking the dictation tests "live" or those tak- 
ing them "recorded.” As measured by standard deviations, the greatest vari- 
ability in age, education, trai ling, and work experience fluctuated from sample 
to sample ior eacn of these characteristics. 
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TABLE 3 



Means, Standard 


Deviations (SD) , and Ranges for 
t tie Ten Experimental Samples 


Age (years) 


for 


Samp le 


N 


Mean 


SD 


Range 


1 


25 


20.5 


2.55 


18.6 - 


29. 2 


2 


35 


20.2 


1.55 


18.0 - 


OJ 

<r 

CM 


3 


29 


19.8 


1.44 


18.1 - 


26.0 


4 


33 


19.7 


1.08 


18.2 - 


23.3 


5 


18 


19.8 


1.16 


18.5 - 


22.9 


6 


34 


19.5 


1.10 


18.1 - 


23.2 


7 


11 


20.4 


1.23 


18.5 - 


22.0 


3 


31 


20.1 


1.59 


18.4 - 


24.3 


9 


14 


21.7 


7.82 


l/.b - 


42. b 


10 


12 


19.2 


1.39 


17.8 - 


22.8 



Means, 


Standard 


TABLE 4 

Deviations (SD) , and Ranges for 
for tbe Ten Experimental Samples 


Education (years) 




Samp le 


N 


Mean 


SD 


Range 




1 


25 


13.680 


1.085 


12 - 


10 




2 


35 


13.829 


1.028 


13 - 


17 




3 


29 


13.586 


1.034 


12 - 


17 




4 


33 


13.424 


. 740 


13 - 


16 




5 


18 


13.500 


.833 


13 - 


16 




6 


2 '4 


13.332 


.687 


13 - 


lb 




7 


U 


14.273 


1.135 


13 - 


16 




8 


31 


13.613 


.748 


n - 


15 




9 


14 


12.5a 


623 


17 - 


14 




10 


12 


72.833 


1.067 


12 - 


15 






i 



15 
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TABLE 5 

Keans, Standard Deviations (SD) , and Ranges for Shorthand Training 
(number of courses) for the Ten Experimental Samples 



Sample 


N 


Kean 


SD 


Range 




1 


25 


1.680 


.67o 


1 


- 


3 




2 


35 


2.457 


.769 


1 


- 


4 




3 


29 


1.759 


.727 


1 


- 


3 




4 


33 


2.576 


.605 


1 


- 


4 




5 


18 


2. 167 


. 764 


1 


- 


3 




6 


34 


2.824 


.706 


1 


- 


4 




7 


11 


1./27 


.862 


1 


- 


4 




8 


31 


2.355 


.650 


1 


- 


3 




9 


14 


2.0/1 


. /03 


1 


- 


4 




10 


12 


2. 167 


.553 


1 


. 


3 





TABLE 6 

Means, Standard Deviations (SD) , and 
Experience (months) for the Ten 


Ranges for Shorthand 
Experimental Sample 


Work 

s 


Sample 


N 


Mean 


SD 


Range 


1 


25 


3.920 


8.859 


0 


- 38 


2 


35 


8.000 


12.495 


0 


- 46 


3 


29 


.724 


2.318 


0 


- 12 


4 


33 


9.273 


11.349 


0 


- 36 


5 


18 


7.833 


13.061 


0 


- 48 


6 


34 


6.765 


8.974 


0 


- 24 


7 


11 


5.091 


7.051 


0 


- 24 


8 


31 


3.129 


5.458 


0 


- 18 


9 


14 


38.214 


90.026 


0 


- 276 


10 


12 


8.917 


17.221 


0 


- 60 
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Tables 7,8,9, and 10 show these data for the combined student samples. 



TABLE 7 

Means, Standard Deviations (SD) , and Ranges for Age (years) for 
Combined Samples of Brigham Young University Students 



Sample 


N 


Mean 


SD 


Range 


One Examiner 


4b 


20.2 


1.49 


18.0 - 24.2 


Two Examiners 


59 


19.9 


1.92 


18. L - 29.2 


All Live 


105 


20. 1 


1.75 


18.0 - 29.2 


All Recorded 


111 


19.9 


1.36 


18.1 - 26.0 


Live -Experience 


50 


20.0 


1.38 


18.5 - 23.3 


Live-No Experience 


55 


20. 1 


2.03 


18.0 - 29.2 


Recorded-Exper ience 


42 


o 

04 


1.41 


18.4 - 23.6 


Recorded-No Experience 


69 


19.7 


1.31 


18.1 - 26.0 



Means, Standard Deviations 
for Combined Samples of 


TABLE 8 

(SD) , and Range3 for Education (yea 
Brigham Young University Students 


rs) 


Samp le 


N 


Mean 


SD 


Range 


Cue Examiner 


46 


13.935 


1.071 


13 - 


17 


Two Examiners 


59 


13.508 


.890 


12 - 


16 


All Live 


105 


13.695 


.996 


12 - 


17 


All Recorded 


111 


13.532 


.847 


12 - 


17 


Live -Experience 


50 


] 3 . 680 


1-028 


12 - 


16 


Live-No Experience 


55 


13*709 


.966 


13 - 


17 


Recorded-Exper ience 


42 


13.619 


.872 


13 - 


16 


Recorded-No Experience 


69 


13.478 


.827 


12 - 


17 



o 




14 - 



TABLE 9 • 

Means, Standard Deviations (SD) , and Ranges for Shorthand Training 
(number of courses) for Combined Samples of Brigham Young University Students 



Sample 


N 


Mean 


SD 


Range 




One Examiner 


46 


2.283 


.851 


1 - 4 




Two Examiners 


59 


2.339 


.895 


1-4 




All Live 


105 


2.314 


.876 


1 - 4 




All Recorded 


111 


2.234 


.747 


1 - 4 




Live-Experience 


50 


2.520 


.854 


1 - 4 




Live-No Experience 


55 


2.127 


.854 


1 - 4 




Recorded-Exper fence 


42 


2.500 


.588 


1 - 3 




Recorded-No Experience 


69 


2.072 


. 786 


1 - 4 





TABLE 10 

Means, Standard Deviations (SD) , and Ranges for Shorthand Work Experience 
(months) for Combined Samples of Brigham Young University Students 



Sample 


N 


Mean 


SD 


Range 


One Examiner 


4b 


7.304 


11.498 


0 

1 

oc 


Two Examiners 


59 


5.559 


9.035 


0 - 38 


All live 


105 


6.324 


10. 225 


0 - 48 


All Recorded 


111 


5.090 


9.389 


00 

<r 

< 

o 


Live-Experience 


50 


13.280 


11.277 


1 - 48 


Live-No Experience 


55 


— 


— 


— 


Re corded -Experience 


42 


13.452 


10.977 


L - 43 


Recorded-No Experience 


69 


— 


— 


— 
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Test Administration Preferences 

Information showing preferences for taking dictation tests was obtained from 
all sample members except one job applicant in Sample 9. These preferences 
indicated whether the examinee preferred taking dictation by ’'recorded 1 * or 
"live” administration and from a male or female voice. ’’No preference" was 
indicated when neither alternative was dominant. 

Some preference fluctuations between ’'recorded” and "live" administration 
occurred among the ten samples (see Table 11). The "no preference" alternative 
was selected by the greatest percentage of Samples 1, 2, 5, 6 and 10. Of the 
two remaining choices, the percentages preferring "live" or "recorded" dictation 
were about equal for Samples 1 and 6, but "recorded" dictation was the secondary 
choice of Samples 2 and 10. "Live" dictation was the secondary choice of Sam- 
ple 5. In Sample 7, predominant preference percentages were about equally di- 
vided between the "recorded" and "no preference" categories. In Samples 3, 4, 

8 and 9 the "no preference" alternative was not the primary choice. "Recorded 11 
dictation tests were administered to Samples 3, 4 and 8. The greatest percen- 
tage of Sample 3 preferred "Live" dictation, but the greatest percentage of 
Sample 4 and the greatest percentage of Sample 8 preferred "recorded" dictation. 
"Live" dictation tests were administered to Sample 9, and the largest proportion 
of this sample preferred this method of administration. 



ERJC 

afliiSifflirjriain 



A grouping of subsamples presented much the same picture. Largest percentages 
were under the "no preference" category wltn no clear cut preference for one 
particular method of administration. Of the combined sample of 241 examinees, 
297. preferred "Live" administration; 307. preferred "recorded” administration 
and 417. indicated "no preference." Table 12 summarizes these combined sample 
preferences for method of dictation test adml ni, t ra t ion. 
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TABLE 11 

Sample Preferences for Method of Dictation Test Administration 



Sample 


N 


Method 


Preference Percentages 


Live 


Recorded 


Either 


1 


25 


Live 


28 


24 


48 


2 


35 


Live 


23 


34 


43 


3 


29 


Recorded 


52 


14 


34 


4 


33 


Recorded 


21 


42 


37 


5 


18 


Recorded 


22 


6 


72 


6 


34 


Live 


32 


30 


38 


7 


11 


Live 


10 


45 


45 


8 


31 


Recorded 


2b 


55 


19 


9 


13 


Live 


62 


0 


38 


10 


12 


Recorded 


8 


25 


67 







TABLE 12 








Combined Sample 


Pre ferences 


for Method of 


Die tation 


Test Admin 


istra tion 


Sample 


N 


Me thod 


Preference Percentages 






Live 


Recorded 


Ei the r 


Student 

(Samples 1,2, 6,7) 


105 


Live 


26 


31 


43 


Student 

(Samples 3,4,5 I 8) 


111 


Recorded 


31 


32 


37 


Combined Student 
(Samples 1-8) 


216 


Live- 

Recorded 


28 


32 


40 


Combined Applicant 
(Samples 9,10) 


25 


Live- 

Recorded 


3b 


12 


52 


Combined Total 
(Samples 1” 10) 


241 


Live- 

Recorded 


29 


30 


4 . 
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Most examinees preferred Caking dictation from a male voice. The only excep- 
tions were Sample 9 which had no job applicant preferring a male voice and 
Sample 10 which had only 3 or 25% of the 12 job applicants preferring a male 
voice. In these two samples the majority had no voice preference. Table 13 
shows the voice preference of each individual sample. 

TABLE 13 

Sample Preferences for Administration of Dictation Exercises 
by Male or Female Voice 



Preference Percentages 

Sample N Voice Male Female Either 



1 


25 


Ha le-Fema le 


80 


0 


20 


2 


35 


Female 


74 


9 


17 


3 


29 


Kale 


/9 


4 


1? 


4 


33 


Male 


76 


3 


21 


5 


18 


Male 


78 


0 


22 


b 


34 


M a le-Fama le 


o2 


3 


35 


7 


?.l 


Female 


73 


0 


27 


8 


31 


Ma le 


91 


3 


6 


9 


13 


Female 


0 


15 


85 


10 


12 


Male 


25 


17 


58 


Grouping 


samples according 


to similarities of 


administration did not < 


:hange 


these preference trends, i 


jf the 21b students 


, 165 or 76% 


preferred a 


male 


voice; 44 


or 21% had no preference, and only 


7 or 3% preferred a female voi< 


Moreover, 


most of tne combined job applicant 


sample (72%) 


did not lave a 


voice preference. For all 


ten samples 70% preferred a ma 


’ e voice, 5% 


pre- 


ierrea a 


female voice, and 


25% had no preference. These 


grouped data 


are 



presented in Table 14. 
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TABLE 14 



Combined Sample Preferences for Administration of Dictation Exercises 
by Male or Female Voice 









Preference 2 


ercentage s 


Samp Le 


N 


Voice 


Ma le 


Female 


Either 


Student-One Examiner 
(Samples 2,7) 


46 


Female 


74 


6 


20 


Student-Two Examiners 
(Samples 1,6) 


59 


Kal *- 
Fcir ale 


69 


2 


29 


S tudent-Live 
(Samples 1 , 2 „ 6, 7) 


105 


Kale- 

Female 


71 


4 


25 


Stud ent- Recorded 
(Samples 3,4, 5, 8) 


111 


Hale 


81 


3 


16 


Combined Student 
(Samples 1-3) 


216 


Male- 
Fena le 


76 


3 


21 


Combined Applicant 
(Samp les 9, 10) 


25 


Female 


12 


16 


72 


Combined Total 
(Samples 1- LO) 


241 


Ma 1 e - 
Female 


70 


5 


25 



Test Performance 

The various subsamples performed differently on the dictation exercises. On 
Exercise No. 4 rounded mean error scores ranged from 5 to b for Samples 4,b 
and 7*, 8 to 9 for Samples 2 and 5; 12 for Sample 8; 17 to 18 for Samples 9 and 
10; and 20 to 24 for Samples 1 and 3. A somewhat simi. * pattern cas shown 
on Exercise No. 10 except for slight improvement by Samples 5 and 1\ a small 
increase in mean errors fa c Samples 4 and fc; and a noti;eable increase in 
errors for Samples 1 and 10, making a rounded mean error score range i .4 to 
28^ Both "live 11 and "recorded" test administrations are represented at most 
of these various Levels of achievement, indicating that sample differences and 
not administration method were largely responsible for fluctuations in achieve- 
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In general, Chose samples with lowest mean error scores also showed least 
variability uf test performance as measured by standard deviations and 
ranges. These data are summarized in Table 15. 

TABLE 15 

Means, Standard Deviations (SD) , and Ranges fur Error Scores Achieved 
on USES Dictation Exercises No. 4 and No. 10 by the Ten Experimental Samples 







Mean 




SD 


Range 


Sample 


N 


Ex. 4 


Ex. 10 


Ex. 4 


Ex. 10 


Ex . 4 


Ex . 10 


1 


25 


24.080 


28.320 


15.515 


18.371 


6-68 


4-75 


2 


35 


9. 257 


9.086 


9.034 


8.016 


0-39 


1-33 


3 


29 


20.276 


20-000 


14.790 


13.807 


4-5/ 


2-48 


4 


33 


5.212 


6.091 


4 . b89 


6.012 


1-21 


0-23 


5 


18 


8. 167 


5.60/ 


8.234 


4.000 


L- 28 


0-L5 


6 


34 


4.559 


6.000 


6.386 


8.967 


0-28 


0-36 


7 


11 


5.818 


4.2/3 


4.933 


3.910 


0-15 


0-13 


8 


31 


12.258 


13.387 


10.746 


11.878 


1-4/ 


1-40 


9 


14 


18.0/1 


1/ 85/ 


IB. / 3 7 


19.654 


1-74 


l-b8 


11 


12 


17.250 


25.333 


18.749 


20.536 


1-66 


4-72 


Separating the ten 


samples 


into student 


and job 


applicant 


groups 


evea led 


the tact that 


performance 


on each exercise was s 


imilar for 


a given 


group . 


This grouping 


also 


showed 


that mean error scutes 


for students were 


con- 


siderabiy lower than mean ' 


error scores 


for job applicants. 


Total 


error 


scores for the two 


exercises ranged from 0 to 74 


for ExercLse No. 


4 and 



from 0 to 75 for Exercise No. 10. Table 16 presents these comparative data. 
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TABLE 16 



Means, Standard Deviations (SD) , and Ranges for Error Scores Achieved on 
USES Dictation Exercises No. 4 and No. 10 by Combined Experimental Samples 



Exercise 4 



Sample 


N 


Mean 


SD 


Range 


BYU 


216 


11.259 


12. 223 


0-68 


PCS 


26 


17.692 


13.747 


1-74 


Total 


242 


11.950 


13. 231 


0-74 



Exercise 10 



Sample 


N 


Mean 


SD 


Kaii at 


BYU 


216 


11.921 


13 . 192 


lT\ 

1 

O 


PCS 


26 


21.308 


20.409 


1-/2 


Total 


24 2 


12. 930 


14.441 


0-75 




When the various BYU student samples were combined along the specific re- 
search dimensions of number of examiners, method of administration, and 
stenographic experience, performance differences were reduced. Table 17 
shows means, standard deviations, and ranges for error scores acnieved 
on Exercises 4 and 10 Or these combined samples. Examination of this 
teble reveals that a _,iven student sample had similar mean scores and simi- 
lar standard deviations on both dictation exercises. 
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TABLE 17 



Keans, Standard Deviations, (SD) , and Ranges for Error Scores Achieved on 
USES Dictation Exercises No. 4 and No# 10 by Combined Samples of 
Brigham Young University Students 



Sample 


N 


Means 


SD 


Range 




Ex. 4 


Ex. 10 


Ex. 4 


Ex. 10 


Ex. 4 


Ex. 10 


Live-One Examiner 


46 


8,435 


7.935 


8.371 


7.534 


0-39 


0-33 


Live-Two Examiners 


59 


12.831 


15.458 


14.733 


17.635 


0-68 


0-75 


All Live 


105 


10.905 


12.162 


12.580 


14.613 


0-68 


0-75 


All Recorded 


111 


11.595 


11.694 


11.864 


11.637 


1-57 


0-48 


Live ‘Experience 


50 


8.320 


9.680 


8.399 


12.791 


0-31 


0-58 


Live-No Experience 


55 


13.255 


14.418 


14 . 944 


15.756 


0-68 


0-75 


Recorded -Experience 


42 


7.624 


5.595 


8.531 


4.440 


1-44 


0-21 


Recorded-No Experience 


69 


14.37 1 


15.406 


12.716 


13.039 


1-57 


0-48 



Differences between mean error r-'ores achieved on the two exercises by eacn 
combined student sample were tested for significance. M t" ratios shoved no 
significant difference in mean performance on the two exercises for any of 
these samples (see Table 18) . On the other hand, there were some significant 
differences in mean performance when the various student samples were com- 
pared. These differences were limited as far as method of administration 
and number of examiners were concerned. On Exercise No. 10 the mean error 
score for the "live 11 group with two examiners was greater than that for tne 
"live" group with one examiner at the .01 level of significance. 
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Significance of Difference between Kean Error Scores Achieved by 
Each Combined Sample of Brigham Young University Students on 
USES Dictation Exercises No. 4 and No. 10 



Sample 


Critical Ratio 


Significance Level 


Live-One Examiner 


-l.b45 


Not Sig. 


Live-Two Examiners 


.462 


Not Sig. 


All Live 


-1.221 


Not Sig. 


All Recorded 


- .i4e 


Not Sig 


Live-Expet ience 


-1.075 


Not Sig. 


Live-No Experience 


- . 734 


Not Sig. 


Recorded -Exp er ience 


1.541 


Not Sig. 


Recorded-No Experience 


-i. 143 


Not Sig. 



More differences existed when the factor of experience was applied. On 
Exercise No. 4 the mean error score for t.ie ‘‘live" group without experience 
was significantly higher rhan the "live" group with experience. Xne mean 
error score on this same test for "recorded" group without experience was 
also significantly higher than the mean error score for the "recorded" group 
with experience. Significance levels for these comparisons were .05 and .01 
respectively, un Exercise No. 10 two significant differences were found whan 
comparing mean performance of experienced and nonexper ienced groups. On the 
average, the recorded sample without experience made s igni t i can t ly more errors 
than the recorded group with experience. The significance level of this dif- 
ference was .01. Also, the "live" group with experience nad a significantly 
higher mean error score (.05 level ot significance) than its recorded counter- 
part. These differences are shown in Table 1^. 
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TABLE 1? 

Significance of Difference between Mean Er > . Scores Achieved by 
Combined Samples of Brigham Young Un- ty Students on 

Dictation Exercises No. 4 and No. 10 



Samp In Comparisun 


Critical Ratio 


significance Level 


Live-One Examiner #4 
vs 

Live-Two Examiners #4 


-1.923 


Not Sig. 


Live-One Examiner #10 
vs 

Live-Two Examiners #10 


-2.950 


.01 


All Live #4 
vs 

All Recorded #4 


- .4’ 4 


Not Sig. 


All Live #10 
vs 

All Recorded #10 


• 239 


Not Sig. 


Live-Experience #4 
vs 

Live-No Experience #4 


-2.090 


.05 



Live-Experience #10 

vs -1 . 698 l >t Sig. 

Live-No Experience #10 



Recordec.-’Txper ience #4 

vs -3.642 ,01 

Recorded-No Experience #4 



Recorded-Exper ience #10 
vs 

Recorded-No Experience #10 


-5.711 


.01 




Live-Experience #4 
vs 

Recorded-Expe r ience #4 


.723 


Not Sig. 




Live- Exper ience #10 
vs 

Recorded-Evper ience #10 


2,112 


.05 




Live-No Experience #4 
vs 

Recorded-No Experience #4 


- .443 


Not Sig. 





Live-No Experience #10 
Q vs 

^ |^|(^ tecorded-No Experience #10 



- .374 
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Not Sig. 
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An analysis of variance £ol the chvee factors of tests, method of administra- 
tion, and experience was made for the total BYU student sample. Table 20 
summarizes the results of this analysis. These data indicate that experience 
is the only factor that exerted any significant influence on shorthand skill. 
The "F" ratio of 17.769 for this factor is significant at the .001 level. 



TABLE 20 

Summary of Analysis of Variance 



Source of Variation 


df 


SS 


MS 


F 


Between subjects 


215 








A (Method) 


l 


69.47961 


69.47961 


■ 262 


B (Experience) 


i 


47H9. 63432 


4709.63432 


17.769* 


AB (interaction 


1 


367 .7* 323 


367.74323 


1.387 


A x B) 










Denominator Terra 


212 


56189.87650 


265.04659 




Within subiects 


215 








C (Tests) 


l 


29.93986 


29.93986 


.089 


AC (Interaction 
A x C) 


1 


55.74268 


55.74268 


.166 


BC (Interaction 
B x C; 


l 


33.39174 


33.39174 


. 100 


ABC (Interaction 


1 


45.564 8 6 


45.56486 


.136 


A x B x 0) 










Denominator Term 


211 


70681.80690 


334-98487 





*Significant at the .001 level 



Some interesting differences in mean performance of student samples are 
evident from these statistical analyses. First of all, there is a definite 
trend for the "live” sample with one examiner to exhibit better average per- 
formance on each dictation exercise than the "3 ive" sample with two examiners. 
The mean difference on Exercise 4 approaches significance at the .05 level. 
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and the mean difference on Exercise 10 is significant at the .01 level. 
Secondly* the method of adminis tration-- n live" or ”recorded"--has little 
bearing on average performance. Last of all, the factor of experience 
exerts a greater influence on test performance than either the test 
form or the method of administration. 



’V* ratios computed for the job applicant 
significant difference in mean performance 
However, the mean number of errors on Exer 
than the mean number of errors on Exercise 
Cross comparisons between the two samples 
in mean performance on Exercise 4 and Exer 



samples indicated there was no 
on the two exercises for Sample 9 
cise 10 w a* 5 significantly greater 
4 at the .01 level for Sample 10. 
revealed no significant difference 
cise 10. 
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Test Performance and Personnel factors 

The relationships between test performance and age, education, shorthand train- 
ing, and shorthand work experience were determined by computing Pearson product- 
moment correlations with erroc scores for each personnel factor. Table 21 shows 
these correlations for the ten experimental samples. Only a few of tnese cor- 
relation coefficients were significant. Age correlated significantly with Sam- 
ple 0 error scores at the .05 level tor Exercise 4 and at the .01 Level for 
Exercise 10. Years of education correlated significantly with error scores 
for Sample 6 at the .01 level for Exercise 10, and with error scores for Sample 
d at the -05 level for Exercise 4. 
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This general lack of significant re la tionsiiip between test performance and age 
and education is not too surprising when one takes into consideration the fact 
that these experimental samples were not radically different as far as means 
and standard deviations for age and education are concerned (see Tables 3 and 4). 
Furthermore, the amount of general education dues not appear to be a critical 
factor after a certain minimum leveL has been reached. 
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TABLE 21 

Pearson Product -Moment Correlations with USES Dictation Test Error Scores 
for Age, Education, Shorthand Training and Shorthand Work Experience for 
the Ten Experimental Samples 



Sample 


N 


(y 

Ex. 4 


Age 

ears) 

Ex . 10 


Educa t ion 
(years) 

Ex. 4 Ex. 10 


Training 
(no . o f courses) 
Ex. 4 Ex. 10 


Experience 

(months) 

Ex. 4 Ex. 10 


1 


25 


.039 


-.17/ 


-.020 


-.316 


-.283 


-.288 


-.226 


-.082 


2 


35 


-.158 


- . °23 


-.171 


. 137 


.0/4 


.021 


-.148 


175 


3 


29 


-.116 


- .139 


-.029 


-.034 


.099 


-.010 


.097 


-.142 


4 


33 


- . 135 


. 045 


-.140 


.134 


.021 


-.081 


-.015 


• 017 


5 


18 


.428 


.211 


.45/ 


.283 


.305 


.309 


. 275 


. 122 


6 


34 


.355* 


.509** 


.314 


.521** 


-.056 


-.009 


-.170 


-.200 


7 


11 


.208 


-.448 


.366 


-.508 


-.311 


.453 


-.224 


.345 


8 


31 


* 253 


.080 


.418* 


.344 


-.129 


-.185 


-.281 


-.415* 


9 


14 


-.223 


- 129 


-.046 


-.017 


-.391 


-.542* 


-.235 


-.142 


10 


12 


-.134 


- .114 


-.119 


-.074 


744**-. 60 7* 


-.124 


-. lbO 



♦Significant at the .05 level 
♦♦Significant at the .01 level 



Significant negative correlations between er' or scores and shorthand training 
and shorthand work experience were also few in number. The number of shorthand 
courses correlated significantly with three of the four error scores for job 
applicant Samples 9 and 10. On Exercise 4 the error scores of Sample 10 corre- 
lated with training at the .01 level of significance, and on Exercise 10 the 
error scores of both Samplec 9 and 10 correlated with training at the .05 level 
of significance. Only one significant negative correlation was found between 
length of shorthand work experience and error scores. This occurred for Sample 8 
error scores on Exercise 10 and was significant at the .05 level. 



o 

ERIC 



30 



27 



The above results indicate that tnere is no consistent relationship between 
test performance and personnel factors for the ten experimental samples. How- 
ever, combining student Samples 1 through 8 according to method and experience 
gave a somewhat different picture. There were no significant correlations be- 
tween age and test performance and only one significant negative correlation 
between education and test performance. The latter was at the .05 level for 
the Exercise 10 performance of the recorded group with no experience. Two 
negative correlations oetw^en work experience and test performance were signi- 
ficant. These were for Exercise 4 for the "all live" sample and for Exercise 
10 for the "all recorded" sample. These negative correlations were significant 
at tne .05 and .01 levels of significance respectively. 

TABLE 22 

Pearson Produc t-homen t Correlations with USES Dictation Test Error Scores for 
Age, Education, Shorthand Training and Shorthand Work Experience for Combined 
Samples of Brigtiam Young University Students 

Age Education Training Experience 

Sample N (years) (years) (no. of cout.es) (months) 







Ex . 4 


Ex, 10 


Ex. 4 


Ex. 10 


Ex. 4 Ex. 10 


Ex. 4 


Ex. 10 


One Examiner 


4b 


122 


-.083 


-.111 


-.009 


.074 


172 


- . 132 


-.099 


Two Examiners 


59 


.244 


. 145 


.157 


.051 


-.519**-. 


,493** 


-.24 2 


- . L91 


All Live 


105 


,133 


.0/2 


.021 


-.025 


-.323**-, 


.296** 


- . 199* 


-.161 


All Recorded 


111 


.083 


.019 


.154 


. 144 


-.148 


.211* 


170 


-.249** 


Live-Experience 


50 


.019 


-.0/6 


. Qb4 


- . Ot b 


-.295* -. 


. 1 0b 


-.201 


-.122 


Live-No Experience 


55 


.170 


. 141 


-.005 


.001 


-.30b* -. 


34 9** 


— 


— 


Recorded-Exper ience 


42 


.111 


-.078 


.133 


.034 


-.120 -. 


.2 42 


. 123 


.139 


Recorded-No Experience 


b9 


. 148 


.133 


.217 


.252* 


-.053 


096 


— 


— 



^Significant at tne .05 level 

**S ignt i leant at tue .01 level 
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The greatest number of significant negative correlations was found for length 
of shorthand training. For tnis variable, correlations which were significant 
at the .01 level occurred between Exercise 4 and Exercise 10 error scores for 
the "live" group having two examiners and for the "all live" group. Negative 
correlations significant at the .05 level were noted for the "all recorded" 
group on Exercise 10 and for the "live-experience" and "live-no experience" 
groups on Exercise 4. Exercise 10 error scores for the "live" group without 
experience also correlated negatively with training at the .0] level of signi- 
ficance. These results which are shown in Table 22 indicate that, although the 
pattern is not consistent across all combined student samples, there is a 
definite trend for students with more shorthand training to make fewer transcrip- 
tion errors. This trend also agrees with corre.la :ion results of the two job 
applicant samples. 

Rel iab ^ lity of Testing Methods 

The cu.jor purpose for conducting this research was to determine whether there 
is a significant difference in the reliability of administering dictation tests 
"Live" or by recording. Recorded administration appears to have the greater 
consistency since the timing and delivery of each dictation exercise are the 
same for each testing session and the role of the examiner is less critical. 
"Live" administration seems to have greater possibilities for incons ir tency be- 
cause each test has to be timed as it is being given, individual examiners may 
vary their delivery slightly from group to group and from day to day, and the 
test may be administered by different examiners all having their own individual 
test administration style. Another contributing factor to unreliability of 
"live" administration is the less frequent administration of proficiency tests 
required in small public employment offices. 
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The apparent reliability advantage of recorded dictation tests is offset by 
certain limitations. Special equipment must be installed in each test center 
to play the recordings, the equipment and installation must provide accept- 
able sound reproduction, and the personal touch of "live 1 ' administration is 
lost. In this study an attempt was made to overcome these deficiencies by 
using good sound equipment and retaining "live 1 ’ administration of test dir- 
ections . 



Reliability of testing methods was determined by computing reliability co 
efficients using error scores on Dictation Exercises 4 and 10. Table 23 
shows these reliability coefficients for the ten experimental samples. 



TABLE 23 

Reliability Coefficients for Live and Recorded Dictation Exercises 
BYU Student Samples 



LIVE 


RECORDED 


One Examiner J 


1 

[ Two Examiners 


Individual Examiners 


Sample 2 


Sample ? 


j Sample l 


Sample C 


Sample 3 


Sample 4 


Sample 5 


Sample 8 


.601 


• 050 | 


1 *520 


.619 


. 789 


.773 


.803 


.7 78 


.577 


. 7 28 










.711 


.820 



PCS Job Applicant Samples 




LIVE 


RECORDED 


Sample 9 


Sample 10 


.862 


. 944 



113 
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It is interesting to note that the coefficients for the "recorded” samples 
are, as a group, larger than the coefficients for the "live" samples; but, 
raw coefficient values can be misleading when determining significant dif- 
ferences in reliability. Consequently, these differen'ss were Analyzed 
by transforming the reliability coefficients to z coefficients, computing 
the standard error of difference between these transformed coefficients 
and then checking the critical ratios between the differences of these co- 
efficients and tneir standard errors for significance. Critical ratios of 
reliability differences for the ten samples are presented in Table 2d. 

TABLE 2d 

Critical Ratios of Differences between Reliability Coefficients for 
Kean Error Scores Achieved by the Ten Experimental Samples on USt' 
Dictation Exercises No. d and No. 10 4 



First Element of Paired Comparisons 


Sample 


1 


2 


3 


d 


5 6 7 


2 


-.397 










3 


-1.692 


-I. d39 








d 


-1.568 


-1.299 


.187 






5 


-1.553 


-1.310 


.0925 


- 253 




6 


- .538 


- . 159 


1.279 


1.132 


1. 176 


/ 


1.284 


1.619 


2. 523* 


2.438* 


2.39S* 1.715 


8 

10 


-1.650 


-1 .391 


.0734 


- .11 d 


.156 -1.227 -2.494* 


^Significant at the .05 




*1 
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These critical ratios indicate that reliability was not affected signifi- 
cantly by metiiou of test administration. Sample 7 was significantly less 
reliable than Samples 3, 4, 5 and 8 at the .05 level. However, the low 
reliability of Sample 7 (.050) may be explained in part by the chance 
fluctuations of a small sample. Although the greatest error score for this 
sample was 15, the complete reversal of error scores (some positive and some 
negative changes) by six of the 11 sample members reduced the reliability co- 
efficient almost to zero- 

Because the "n's" for each of the ten experimental samples were small, relia- 
bility coefficients were also computed for the combined samples of Brigham 
Young University students. These reliability data are included in Table 23. 

This table shows that Sample 7 influenced the reliability of the "live" group 
with one examiner, resulting in a reliability coefficient of .577 compared 
with one of .728 for .he "live" group with two examiners. However, the dif- 
ference between these reliability statistics is not significant. 

The same trend for greater stability of recorded dictation tests noted for 
individual samples perssted for the combined samples. The two reliability 
coefficients are .711 and .820 for the "all live" and "all recorded" groups 
respectively. The n t" ratio for the difference between these two groups is 
1.95 which is close to the 1.96 value required for the .05 level of significance. 

Table 25 shows critical ratios end significance levels for these reliability 
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coefficient comparisons. 



32 



TABLE 25 

Significance of Difference between Reliability Coefficients of Mean Error 
Scores Achieved on USES Dictation ExercisesNo. 4 and No. 10 by Combined 
Samples of Brigham Young University Students 



Sample Comparison 


Critical Ratio 


Significance Level 


Live -One Examiner 
vs 

Live-Two Examiners 


-1.332 


Not Sig. 


All Live 
vs 

All Recorded 


-i* 954 


Not Sig. 



A final look at '’live 11 and "recorded" test reliability for student samples 
was made by adding the factor of experience. Reliability coefficients of 
"live" and "recorded" dictation tests administered to students with short- 
hand work experience and to students without shorthand work experience appear 
in Table 26. 

TABLE 26 

Reliability Coefficients for Live and Recorded Dictation Exercises 
Administered to Brigham Young University Students Classified 
According to Experience 



LIVE 


RECORDED 


Experience 


No Experience 


Exper i °nce 


No Experience 


.717 


.708 


,74b 


.832 











No significant difference was found between the reliability of the "live" 
group with experience and the "live" group without experience. For these two 
groups the coefficients were .717 and .708. Although the reliability coeffi- 
cient for the recorded group without experience (.832) was greater than the 
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reliability coefficient for the recorded group with experience (.746), the 
difference between these coefficients was not significant. Comparisons 
between the "live" and "recorded 1 * groups with experience and the M live ,T and 
"recorded" groups without experience also showed no significant differences 
in reliability. Table 27 summarizes the critical ratios of these reliability 
comparisons . 



TABLE 27 

Significance of Difference between Reliability Coefficients for Mean 
Error Scores Achieved on USES Dictation ExercisesNo. 4 and No. 10 
by Combined Samples of Brigham Young University Students 
Classified According to Experience 



Sample Comparison 


Critical Ratio 


Significance Level 


Live -Experience 
vs 

Live-No Experience 


.099 


Not Sig. 


Record ed-Fxpe r ie nee 
vs 

Recurded-No Experience 


-X .089 


Not Sig. 


Li vc -Exp er ience 
vs 

Recorded -Experience 


- .27/ 


Not Sig. 


Live-No Experience 
vs 

Recorded-Wo Experience 


-1.618 


Not Sig. 



ihfi above analysis snows that administration of dictation tests by the "recorded' 
method is fu* f y as reliable ns administration by the "liv^' r method. In fact, 
reliability coefficients of "recorded" administrat ions are larger, although not 
significantly so, than those of "live :I administrations. 
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SUMMARY AND CONCLUSIONS 



This research was conducted to determine the comparative reliability of 
"live" and "recorded ,r dictation tests. The total experimental sample .of 
242 examinees was drawn from (1) University students and (2) local office 
job applicants. Each sample had only one male participant. The 216 stu- 
dents were enrolled in a second semester shorthand course at Brigham Young 
University, Provo, Utah. Most of the instruction in this course was given 
by female instructors, but recordings with a male voice were used regularly 
in the teaching process. The 26 job applicants were applying for stenograpn- 
ic positions through the Professional, Clerical, and Sales Office of the 
Utah Department of Employment Security, Salt Lake City, Utah. The average 
member of the student sample was 19.9 years of age, had 13.6 years of educa- 
tion, had taken 2.3 courses in shorthand, and had had 5.7 months of shorthand 
work experience. The average member of the job applicant sample was .7 years 
older, had .9 years less education, had completed .2 fewer shorthand courses, 
and had had more chan four tiroes as much on- job shorthand experience. 

The experimental test battery included USES Dictation Exercise No. 4 and 
USES Dictation Exercise No. 10 taken from the G uide to the Use of Typing , 
Dictation and Spe 1 1 Ing Tests , published by thi Bureau of Employment Security 
in December, 1953. These exercises were dictated at 80 words per minute 
either "live" or by tape recording. The same standard tape recording with a 
male voice was used for "recorded" administrations. Standardized test 
directions were given "live" from the Guide . Supplementary instructions were 
given to clarify "recorded" administration. A brief introduction stating the 
purpose of testing and a practice exercise preceded the two test exercises. 
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Test performance was measured by number of transcription errors. Transcrip- 
tions were scored according to "Directions for Scoring Transcription" appear- 
ing in the Guide . Personnel information including age, education, amount of 
shorthand training, and length of shorthand work experience along with pre- 
ferences for method of test administration and sex of test administrator were 
obtained through a special Information Sheet. Personnel data were collected 
from all members of the experimental sample, but preference data were lacking 
fo* one member of the job applicant sample. 

Test data were collected on Hay 11 and 12, 1966 for the student sample. Stu- 
dents were tested in eight groups, ranging in size from 11 to 35 examinees. 

Four of these groups (105 students) were administered the test battery "live." 
The other four groups (111 students) were administered the test battery by tape 
recording. The "live" groups had either one or two examiners administering 
the tests to them, and the "recorded" groups had only one examiner administer- 
ing tne tests. Both male and female examiners were represented. Each exercise 
was full) transcribed after its dictation. Standard transcription time was 
reduced from 20 to 15 minutes for each exercise to acconsnudate the 50-minute 
class period. 

Job applicants were tested from June 6 to 23, 1966. Two female examiners 
administered the tests to this sample in groups of one to four examinees. 

Metnod of administration was changed randomly from day to day. Fourteen 
members of tne job applicant sample were given the test "live," and 12 ware 
given the test by recording. Sometimes on n examiner gave both tests; and 
sometimes both examiners assisted, each examiner administering one of the 
test exercises. Examinees in the job applicant sample were allowed the 
standard 20 minute period to transcribe each test exercise. 
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Test data were analyzed individually for the eight student subsattiples and the. 
two job applicant subsamples. They were also analyzed collectively for tne 
combined student and combined job applicant samples. Testing preferences 
were analyzed not only for these samples but also for the combined totr.l sample. 
This distinction in analytical comparison was made because of basic differences 
in personnel characteristics and testing conditions between the student and 
job applicant samples. 

Quantitative analysis included computation of means, standard deviations, 
ranges, Pearson product-moment correlations, reliability coefficients, stan- 
dard errors of differences between means and between reliability coefficients, 
analysis of variance, and t and F ratios. The comparing of so many differ- 
ences between small samples through the use of M t M tests may be a questionable 
technique, but the findings from this procedure -re supported by the results 
of analysis of variance. 

The analysis of student and job applicant data supports the following con- 
clusions: 

1. Either "live" or “recorded" dictation test administration is 
acceptable to examinees since no clear-cut administration 
preference was expressed by either students or job applicants. 
Combined total sample preference percentages were 29 for "live,” 

30 for "recorded," and 41 for either method. 

2. Most examinees prefer taking dictation from a male voice. For 
all ten samples 70% preferred a male voice, 5% preferred a 
female voice, and 25% had no preference. 

3. Mean error scores on each dictation exercise were generally 
similar for a given subsaraple. They were also similar for a 
given group when student samples were combined along specific 
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research dimensions and when the ten subsamples were separated 
into student and job applicant samples. 

4. Although students had less transcription time than job applicants, 
their mean error scores were considerably lower. Respective means 
were 11.3 and 17.7 for Exercise No. A and 11.9 and 21.3 for 
Exercise No. 10. 

5. Differences between mean error scores achieved on the two exercises 
by each combined student samples were not significant. However, 
the mean error score on Exercise No. 10 was significantly higher 
than the mean error score on Exercise No. A for Applicant Sample 10. 

6. There were some significant differences in mean pe\formance for 
cross comparisons of combined student samples but not for job 
applicant samples. These differences were limited fur method of 
administration but were more numerous when samples were clarified 
according to experience. 

7. Menn student perfo r mance of the "recorded" group with experience 
was significantly better on the two exercises than the moan per- 
formance of the "recorded 11 group without experience. Both of 
these differences are significant at the .01 level. Mean student 
performance of the "live" group with experience was significantly 
better on Exercise No. A than the mean performance of the "live" 
group without experience (.05 level of significance) . 

6. An analysis of student sample variance verifies the fact that 
experience ratner than test form or method of administration 
exerted the greatest significant influence on snorthand skill. 

9. As measured hy Pearson produc t -moinen t correlations, there is n^ 

consistent re lal lonsnip between test performance and tne pt-rsmiicl 
factors of age, education, amount of shorthand training, and 
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and length of shorthand experience for tbe ten experimental 
samples. However, for combined "live” student samples there 
is a trend for those with more shorthand training to perform 
better on the dictation exercises. 

10. Reliability coefficients are larger for individual subsamples 
administered the test battery by recording than for subsamples ad 
ministered the test battery ’’live;" but the only significant 
differences in reliability occurred between Student Sample 7 
(Live-One Examiner) and Student Samples 3, 4, 5, and 8 (Recorded) 
The low reliability of Sample 7 ( .050) can be explained by the 
chance fluctuations in performance of a small sample (N=ll) • 
Consequently, these comparative statistics are of little import. 

11. Larger reliability coefficients favoring the "all recorded” 
stjdent sample over the “all live” student sample, the "live" 
student sample with two examiners over the "live 1 ' student sample 
with one examiner, and the "recorded" applicant sample over the 
"live lr applicant sample were not statistically significant. 

12. There were no significant differences in reliability among the 
"live" and "recorded" student samples when tnese were classified 
according to shorthand experience. 

13. Although student examinees nad tne advantage or being familiar 
with recorded dictation tests, the administration of dictation 
tests by the "recorded" method appears to be fully as reliable 
as administration by the "live" method. 
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